Goto

Collaborating Authors

 inhibited softmax


Inhibited Softmax for Uncertainty Estimation in Neural Networks

arXiv.org Machine Learning

We present a new method for uncertainty estimation and out-of-distribution detection in neural networks with softmax output. We extend softmax layer with an additional constant input. The corresponding additional output is able to represent the uncertainty of the network. The proposed method requires neither additional parameters nor multiple forward passes nor input preprocessing nor out-of-distribution datasets. We show that our method performs comparably to more computationally expensive methods and outperforms baselines on our experiments from image recognition and sentiment analysis domains. The applications of computational learning systems might cause intrusive effects if we assume that predictions are always as accurate as during the experimental phase. Examples include misclassified traffic signs (Evtimov et al., 2018) and an image tagger that classified two African Americans as gorillas (Curtis, 2015). This is often caused by overconfidence of models that has been observed in the case of deep neural networks (Guo et al., 2017). Such malfunctions can be prevented if we estimate correctly the uncertainty of the machine learning system.